Hearing protection and communication apparatus using vibration sensors

ABSTRACT

Hearing protection and communication apparatus using vibration sensors are disclosed. An example wearable electronic device includes means for transducing vibrations associated with speech into a first signal; means for transducing sound associated with ambient noise into a second signal; and means for processing to cause a speaker to output a signal to reduce the ambient noise; detect an identifier in the speech; and cause audio data representative of the speech to be transmitted to a second device associated with the identifier.

RELATED APPLICATION

This patent arises from a continuation of U.S. patent application Ser.No. 15/912,006, now U.S. Pat. No. 11,007,081, which was filed on Mar. 5,2018. U.S. patent application Ser. No. 15/912,006 is hereby incorporatedherein by reference in its entirety. Priority to U.S. patent applicationSer. No. 15/912,006 is hereby claimed.

BACKGROUND

Hearing protection devices may be used to protect ears from damage tothe tympanic membrane or the nerves of the cochlea in the ears. Forexample, such hearing protection devices may include ear muffs, earplugs, or headphones with noise cancellation technology.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an example system forcommunication and ear protection in noisy environments;

FIG. 2 is a flow chart illustrating an example process for communicationand ear protection in noisy environments;

FIG. 3A is a diagram illustrating an example graph of a raw signalincluding speech captured from a vibration sensor;

FIG. 3B is a diagram illustrating an example graph of a raw signalincluding speech captured from a microphone in a noisy environment;

FIG. 3C is a diagram illustrating an example graph of a processed signalincluding speech captured from a vibration sensor in a noisyenvironment;

FIG. 3D is a diagram illustrating an example graph of a processed signalincluding speech captured from a microphone in a noisy environment;

FIG. 3E is a diagram illustrating an example graph of another processedsignal including speech captured from a vibration sensor in a noisyenvironment;

FIG. 3F is a diagram illustrating an example graph of another processedsignal including speech captured from a microphone in noisy environment;

FIG. 4 is a flow chart illustrating a method for integrated hearingprotection and communication;

FIG. 5 is a flow chart illustrating a method for processing speechcaptured via vibration sensors;

FIG. 6 is a flow chart illustrating a method for processing ambientsound;

FIG. 7 is block diagram illustrating an example computing device thatcan communication and ear protection in noisy environments; and

FIG. 8 is a block diagram showing computer readable media that storecode for providing communication and ear protection in noisyenvironments.

The same numbers are used throughout the disclosure and the figures toreference like components and features. Numbers in the 100 series referto features originally found in FIG. 1; numbers in the 200 series referto features originally found in FIG. 2; and so on.

DESCRIPTION OF THE EMBODIMENTS

As discussed above, hearing protection devices may be used to protectears from injuries caused by exposure to environments with high levelsof noise. For example, workers in high noise environments of 85 decibelsor louder may wear ear muffs or ear plugs to prevent hearing lossassociated with extended exposure to high levels of noise. In someexamples, users may use passive or active hard shell protectors or foamtype inserts in the ear canal. However, using hearing protectors mayalso prevent effective communication between users because all soundsare equally diminished by these devices. Moreover, some workers may notuse the hearing protection devices in order to effectively communicate.For example, workers may take off the ear muffs or ear plugs in order tohear another worker yelling over a high level of noise.

The present disclosure relates generally to techniques for communicationand hearing protection. Specifically, the techniques described hereininclude an apparatus, method and system for communication and hearingprotection in high noise environments. An example apparatus includessafety glasses including a vibration sensor to capture speech from auser. The apparatus includes hearing protectors communicatively coupledto the safety glasses and one or more other devices. The hearingprotectors are to reduce a volume of an ambient sound and play backcaptured speech from the one or more other devices.

The techniques described herein thus enable users to communicate clearlyin high noise environments and also be aware of their surroundings. Atthe same time, the techniques described herein also provide effectivehearing protection. Moreover, the techniques may provide increasedawareness of an environment. For example, the techniques describedherein may be used to generate notifications based on particular ambientsounds detected in an environment. In addition, the techniques describedherein may allow automated communication with other devices using speechcommands. For example, speech captured using vibration sensors may beautomatically sent to one or more other devices based on one or moredetected keywords in the speech.

FIG. 1 is a block diagram illustrating an example system forcommunication and ear protection in noisy environments. The examplesystem is referred to generally by the reference number 100 and can beimplemented in the computing device 700 below in FIG. 7 using theprocess 200 or methods 400-600 of FIGS. 2 and 4-6 below.

The example system 100 includes an integrated hearing protection andcommunication system 102 communicatively coupled via a wirelessconnection 106 to one or more other systems 104. For example, thewireless connection 112 can be a short-range wireless connection such asa Bluetooth®, Wi-Fi®, or cellular connection. In some examples, theother systems may include other integrated hearing protection andcommunication systems, automated speech recognition (ASR) systems, andnatural language processing (NLP) systems, among other external systems.The integrated hearing protection and communication system 102 includeshearing protectors 108 and safety glasses 110 communicatively coupledvia a wireless connection 112. For example, the wireless connection 112can be a short-range wireless connection such as a Bluetooth®, Wi-Fi® orcellular connection. The hearing protector 108 includes at least onespeaker 114 and at least one microphone 116. The safety glasses 110include a nose pad 118. The nose pad 118 includes vibration sensors 120.For example, the vibration sensors 120 can be piezoelectric elements oraccelerometers.

As shown in FIG. 1, an integrated hearing and protection andcommunication system 102 can be used to protect ears from high noiseenvironments while allowing communication with one or more other systems104. For example, the hearing protector 108 can reduce the volume ofambient sounds in a high noise environment. The vibration sensors 120 ofthe safety glasses 110 can be used to capture speech of a user wearingthe integrated hearing protection and communication system 102. Forexample, the vibration sensors 120 can capture vibrations of a user'snose when the user speaks. In some examples, the integrated hearing andprotection and communication system 102 can be paired before use. Forexample, the hearing protector 108 can be paired with the safety glasses110 via the wireless connection 112 before the use of the integratedhearing protection and communication system 102. In some examples, theintegrated hearing protection and communication system 102 may alsoreceive a unique identifier from the user. For example, the uniqueidentifier may be in the form of a name or a personal identificationnumber. In some examples, the integrated hearing protection andcommunication system 102 can then use the unique identifier foridentification when connecting to other systems 104, such as nearbytrusted devices or networks, via the wireless connection 116.

As shown in FIGS. 3A and 3C below, vibration sensors 120 can capturespeech in noisy environments with reduced effects from ambient sounds.In some examples, the captured speech can be processed to removenonlinear distortion based on a detected active voice call, as describedin greater detail below. For example, speech that has an active voicecall with a destination of a second integrated hearing protection andcommunication system 104 can be processed using a human-to-human voicetransformation that enables another speaker to understand the speech inaddition to being processed using an ASR voice transformation thatimproves the detectability of words and keywords in the speech. In someexamples, speech that is captured without any detected active voice callcan be processed using an ASR voice transformation that improves thedetectability of words and keywords in the speech and be sent forprocessing at an ASR system 104. The processed speech can then be sentas input into an ASR engine. In some examples, the safety glasses 110may include a speech interface. For example, the speech interface mayinclude a local keyword recognition or an ASR module.

In some examples, the safety glasses 110 may be communicatively pairedwith a high quality hearing protectors. For example, the hearingprotector 108 may include small speakers on the inside for playback ofspeech, ambient sounds, and notifications. In some examples, the hearingprotector 108 may also include microphones 116 mounted on the outside ofthe protector to captured ambient sounds. In some examples, the hearingprotector 108 may include on-board processing for audio event detection,loudness equalization, and sound logging for analysis of long termexposure to high volume environments, as described in detail withrespect to FIG. 6 below.

The diagram of FIG. 1 is not intended to indicate that the examplesystem 100 is to include all of the components shown in FIG. 1. Rather,the example system 100 can be implemented using fewer or additionalcomponents not illustrated in FIG. 1 (e.g., additional integratedhearing protection and communication systems, speakers, microphones,vibration sensors, safety glasses, hearing protectors, etc.).

FIG. 2 is a flow chart illustrating an example process for communicationand ear protection in noisy environments. The example process isgenerally referred to by the reference number 200 and can be implementedin the system 100 above or the computing device 700 below.

At block 202, a processor within a set of safety glasses captures speechvia at least one vibration sensor, transforms the processed speech basedon whether an active voice call is detected, and receives responses tothe processed speech.

At block 204, a processor within a hearing protector captures ambientsounds from a microphone and processes the ambient sounds, along withspeech from the safety glasses and other devices.

At block 206, a vibration sensor captures vibrations from around, near,or at a user's nose and generates a vibration signal 208. For example,the vibration signal 208 may include speech from a user. The vibrationsignal 208 may then be filtered for conversation or ASR voicetransformation using on-board signal processing at blocks 210 and 212.

At block 210, a processor processes the vibration signal 208 with ahuman-to-human voice transformation such that the speech can bedeciphered by other users when played on speakers.

At block 212, the processor processes the vibration signal 208 with anASR voice transformation for improved detection of words, includingkeywords. For example, a user can state command phrases, such as “talkto John” to initiate a conversation with a colleague. In some examples,the safety glasses may have on-board keyword and small vocabularycommand recognition for such tasks. In some examples, large vocabularyASR and NLP can reside on the safety glasses, or be enabled through acloud connection. In some examples, the system can respond back to theuser using text-to-speech (TTS).

At block 214, the processor transmits the vibration signal 208 processedwith a human-to-human voice transformation to the hearing protectors ofa colleague or other users. For example, the hearing protector of theother users may play back the processed vibration signal 208 on one ormore speakers. In some examples, the safety glasses and the hearingprotectors may be paired before use.

At block 216, the processor may receive speech in response to thetransmission as part of a conversation between a user and one or moreother users. For example, when a user wants to initiate a conversationwith a colleague, the user may state: “John, do I need to go left orright?” The glasses can capture the speech signal, and the keywordrecognizer can recognize the identifier “John.” A connection can beestablished between the user and the user “John” and processed speechaudio transmitted to the apparatus associated with the identifier“John.” These signals can then be played back through the hearingprotector speakers of the device associated with the identifier “John.”In some examples, the device associated with the identifier “John” maycaptured and process speech and return processed speech audio to theprocessor in response. The processor may then send the received speechto the paired hearing protector for playback.

As shown at block 222, the hearing protectors can be equipped with oneor more microphones. For example, the microphones may be located on theoutside of the hearing protectors to capture ambient sounds 224.

At block 226, these input ambient sounds 224 can be loudness equalizedto a target volume that is sufficiently low as to not cause any risk ofhearing loss. However, the target volume may be loud enough such thatcues in the ambient environment may also be preserved.

At block 228, the low volume version of the ambient sounds 224 can thenplayed back to the user via one or more speakers 230. In some examples,the loudness compensation can be performed digitally on board. Forexample, the loudness compensation can be performed using a digitalsignal processor in a device. In some examples, the loudness level canbe preset or user adjustable.

At block 232, the processor can characterize and log incoming ambientsounds 224. For example, the ambient sounds can be characterized interms of ⅓ octave levels over time, A-weighted levels, or raw samples.In some examples, the logged ambient sounds can be used to provide meansto measure noise exposure of users. In some examples, the characterizedambient sounds can be used to assess risks and can also be used tocorrelate ambient sounds to events.

At block 234, the processor can characterize incoming ambient sounds 224with an audio event classifier. For example, an audio event classifiercan recognize target sounds in the environment that may be of interest.In some examples, the audio event classifier may be a neural networktrained to detect any number of audio events.

At block 236, the processor logs audio events and generatesnotifications based on the audio events. For example, the notificationsmay correspond to particular sounds that are detected. In some examples,when such sounds are recognized, the system can notify the user. Forexample, the processor can play back a notification of a safety hazard.Thus, the hearing protector may suppress outside noise in most cases,but if there are any particularly sounds that correspond dangeroussituations then the user may be alerted accordingly. For example, suchdangerous situations may include fire alarms, explosions, collisions,etc.

At block 238, the processor mixes different audio together for playback.For example, the different audio may include TTS responses from block220, feedback speech from block 216, notifications from block 236, andreduced volume ambient sounds from block 226. The processor may playback the mixed audio 228 via the speakers 230. For example, the speaker230 may be integrated into hearing protectors.

This process flow diagram is not intended to indicate that the blocks ofthe example process 200 are to be executed in any particular order, orthat all of the blocks are to be included in every case. Further, anynumber of additional blocks not shown may be included within the exampleprocess 200, depending on the details of the specific implementation.

FIG. 3A is a diagram illustrating an example graph of a raw signalincluding speech captured from a vibration sensor. The example graph isgenerally referred to by the reference number 300A.

FIG. 3A shows an example raw signal waveform 302A of speech as capturedin a noisy environment via a vibration sensor using a left channel. Aset experiments were conducted with a prototype device to demonstratebenefits. First, accurate binaural recordings were made in variousenvironments with a Head and Torso Simulator (HATS) system and abinaural headphone. For example, the environments included noise in datacenters, an industrial Fab, a Titanium industrial facility, andconstruction noise. The noise levels in these environments ranged from80 to 110 decibels, A-weighted (dBA). Two example environments wereselected for prototype tests: data center noise with levels ofapproximately 94 dBA, and factory heavy machinery noise of approximately92 dBA. Note that these sound levels are extremely high and requirehearing protection per federal regulations.

In the lab, a prototype system was constructed with safety glasses withintegrated piezo electric vibration sensing elements, hearingprotectors, and a data acquisition system. In addition to the vibrationsensors, data was also simultaneously collected from regular microphonesand 8 channel microphone arrays in the near field and far field. Thesystem was located in a usability lab, and a playback speaker system wasused to generate the high noise scenarios outlined above. A participantwas asked to state a set of utterances that appeared on a screen, andthe data from the sensors was captured. The utterances included a set ofwake up words, personal assistant type utterances, and large vocabularydictation type text.

In the particular examples of FIGS. 3A-3F, the environment included datacenter conditions with an ambient noise of ˜94 dBA and the utterancespecifically used was “Hello Computer.” As can be seen in FIG. 3A,although ambient sounds are present, the captured speech can be clearlydetected at a time of about 2 seconds onwards. The results of anexperiment using an integrated solution with the vibration sensingelements of the techniques described herein demonstrate improvedcommunication in noisy environments. For example, a +10 dBsignal-to-noise ratio (SNR) was observed using vibration sensors withoutany additional processing. Thus, using the techniques described herein,speech can be clearly understood even in these extreme high noiseenvironments.

FIG. 3B is a diagram illustrating an example graph of a raw signalincluding speech captured from a microphone in a noisy environment. Theexample graph is generally referred to by the reference number 300B.

FIG. 3B shows an example waveform 302B of speech as captured in a noisyenvironment via a microphone in a near field using a right channel. Forexample, the captured audio is the same time period as the audiocaptured in the left channel in FIG. 3A above. By contrast to FIG. 3A,as can be seen in FIG. 3B, the captured speech 302B cannot bedistinguished from the ambient sounds, making the speech more difficultto be detected and processed. For a regular microphone, the data centerhigh noise scenario sound-to-noise ratio (SNR) was about −20 dB. Thus,it is very difficult, even for humans, to decipher any speech in theaudio signal.

FIG. 3C is a diagram illustrating an example graph of a processed signalincluding speech captured from a vibration sensor in a noisyenvironment. The example graph is generally referred to by the referencenumber 300C.

FIG. 3C shows an A-weighted Fast Fourier transform FFT versus time ofthe raw audio signal of speech captured in FIG. 3A above. As can be seenin FIG. 3C, the speech 302C can be clearly distinguished from thebackground ambient sounds.

FIG. 3D is a diagram illustrating an example graph of a processed signalincluding speech captured from a microphone in a noisy environment Theexample graph is generally referred to by the reference number 300D.

FIG. 3D shows an A-weighted Fast Fourier transform FFT versus time ofthe raw audio signal of speech captured in FIG. 3B above. As can be seenin FIG. 3D, the speech 302D can be no longer clearly distinguished fromthe background ambient sounds. Thus, audio captured via microphonesperforms worse than audio captured via vibration sensors even withtransformations applied.

FIG. 3E is a diagram illustrating an example graph of another processedsignal including speech captured from a vibration sensor in a noisyenvironment. The example graph is generally referred to by the referencenumber 300E.

FIG. 3E shows an A-weighted level versus time of the raw signal ofspeech captured in FIG. 3A above. As can be seen in FIG. 3E, the speech302E can be clearly distinguished from the background ambient sounds bythe increased A-weighted level at around 2 seconds onwards.

FIG. 3F is a diagram illustrating an example graph of another processedsignal including speech captured from a microphone in noisy environment.The example graph is generally referred to by the reference number 300F.

FIG. 3F shows an A-weighted level versus time of the raw signal ofspeech captured in FIG. 3B above. As can be seen in FIG. 3F, the speech302F cannot be clearly distinguished from the background ambient sounds.In particular, there is no increased A-weighted level at around 2seconds onwards. Rather, the A-weighted level remains about the samethroughout the graph 300F.

FIG. 4 is a flow chart illustrating a method for processing andtransmitting speech captured via vibration sensors. The example methodis generally referred to by the reference number 400 and can beimplemented in the system 100 of FIG. 1 above, the processor 702 of thecomputing device 700 of FIG. 7 below, or the computer readable media 800of FIG. 8 below.

At block 402, a processor captures speech from one or more vibrationsensors and ambient sounds from one or more microphones. For example,the vibration sensors may be piezoelectric sensors or accelerometers inthe nose pad of an integrated hearing protection and communicationapparatus.

At block 404, the processor processes the captured speech based on anactive voice call. In some examples, the processor can process thespeech based on a determination of whether an active voice call isdetected or not detected. For example, the processor can process thespeech using the method 500 described below with respect to FIG. 5.

At block 406, the processor processes ambient sounds to generate ambientsounds with a lower volume and sound events. In some examples, theprocessor can detect sound events from the ambient sounds and generatenotifications based on the sounds events. For example, the processor canprocess the ambient sounds using the method 600 described below withrespect to FIG. 6.

At block 408, the processor transmits processed speech to one or moreother devices and receives processed speech from one or more otherdevices. In some examples, the processor can transmit processed speechto a device based on a detected keyword. For example, one or moredevices may be associated with the detected keyword.

At block 410, the processor plays back processed ambient sounds,processed speech from other devices, and sound events. For example, theprocessed ambient sounds, processed speech from other devices, and soundevents can be combined and played back at one or more speakers ofhearing protectors in an integrated hearing protection and communicationapparatus.

This process flow diagram is not intended to indicate that the blocks ofthe example process 400 are to be executed in any particular order, orthat all of the blocks are to be included in every case. Further, anynumber of additional blocks not shown may be included within the exampleprocess 400, depending on the details of the specific implementation.

FIG. 5 is a flow chart illustrating a method for processing andtransmitting speech captured via vibration sensors. The example methodis generally referred to by the reference number 500 and can beimplemented in the system 100 of FIG. 1 above, the processor 702 of thecomputing device 700 of FIG. 7 below, or the computer readable media 800of FIG. 8 below.

At block 502, a processor captures speech via one or more vibrationsensors. For example, the vibration sensors may be located on the headof a speaker and can capture vibrations generated during the speech ofthe speaker.

At block 504, the processor determines whether a voice call is active.If the processor detects that the voice call is not active, then themethod may proceed at block 506. If the processor detects that a voicecall is active, then the method may proceed at blocks 506 and 508.

At block 506, the processor processes the captured speech with anautomatic speech recognition (ASR) voice transformation. For example,the ASR voice transformation may be optimized for detection of keywordsand phrases ASR applications.

At block 508, the processor processes the captured speech with ahuman-to-human voice transformation. For example, the processor mayprocess the captured speech such that the speech is easier to understandby a human listener.

At block 510, the processor performs a keyword or ASR recognition on theprocessed speech and sends the recognized speech to an NLP system. Forexample, the NLP system can generate one or more TTS responses inresponse to receiving the recognized speech.

At block 512, the processor receives an NLP system TTS response. Forexample, the NLP system TTS response may include answers to questions,messages, notifications, etc.

At block 514, the processor causes the processed speech to betransmitted to one or more other headphones. For example, the processorcan cause a transmitter to transmit the processed speech to a particularset of headphones based on a detected keyword associated with the set ofheadphones. In some examples, a number of devices may be associated witha particular keyword.

At block 516, the processor receives processed speech feedback from oneor more users. For example, the processed speech feedback may includeresponses to the processed speech transmitted at block 514 above.

This process flow diagram is not intended to indicate that the blocks ofthe example process 500 are to be executed in any particular order, orthat all of the blocks are to be included in every case. Further, anynumber of additional blocks not shown may be included within the exampleprocess 500, depending on the details of the specific implementation.

FIG. 6 is a flow chart illustrating a method for processing ambientsound and playing back a mix of speech and ambient sound. The examplemethod is generally referred to by the reference number 600 and can beimplemented in the system 100 of FIG. 1 above, the processor 702 of thecomputing device 700 of FIG. 7 below, or the computer readable media 800of FIG. 8 below.

At block 602, a processor captures ambient sounds and receives an NLPsystem TTS response or processed speech feedback. For example, theambient sounds may be received from a microphone. The NLP system TTSresponse may be received from an NLP system. For example, the NPL systemmay have generated a TTS response as described above with respect toFIG. 5. In some examples, the processed speech feedback may be receivedfrom one or more external devices.

At block 604, the processor detects audio events in the captured ambientsounds and logs the audio events in an event log and generatesnotification sounds based on the audio events. In some examples, thenotification sounds may be based on a detected type of the audio event.

At block 606, the processor equalizes a loudness of the captured ambientsounds based on a target loudness. For example, the target loudness maybe an adjustable volume level that is received from a user. In someexamples, the target loudness may be a present value.

At block 608, the processor characterizes and logs the captured ambientsounds. For example, the captured ambient sounds can be characterizedbased on one or more features of the captured ambient sounds. In someexamples, the captured ambient sounds can be characterized using apretrained neural network. For example, the neural network can betrained using a training dataset including a variety of different typesof sounds. In some examples, the characterized sound can then be logged.For example, the logged characterized sound can be used later to analyzean environment such as a work environment.

At block 610, the processor plays back notification sounds, equalizedambient sounds, an NLP system TTS responses, processed speech feedbackfrom one or more users, or any combination thereof. For example, thenotification sound may correspond to one or more detected sounds in thecaptured ambient sound.

This process flow diagram is not intended to indicate that the blocks ofthe example process 600 are to be executed in any particular order, orthat all of the blocks are to be included in every case. Further, anynumber of additional blocks not shown may be included within the exampleprocess 600, depending on the details of the specific implementation.

Referring now to FIG. 7, a block diagram is shown illustrating anexample computing device that can provide communication and earprotection in noisy environments. The computing device 700 may be, forexample, a laptop computer, desktop computer, tablet computer, mobiledevice, or wearable device, among others. In some examples, thecomputing device 700 may be an integrated hearing protection andcommunication device. The computing device 700 may include a centralprocessing unit (CPU) 702 that is configured to execute storedinstructions, as well as a memory device 704 that stores instructionsthat are executable by the CPU 702. The CPU 702 may be coupled to thememory device 704 by a bus 706. Additionally, the CPU 702 can be asingle core processor, a multi-core processor, a computing cluster, orany number of other configurations. Furthermore, the computing device700 may include more than one CPU 702. In some examples, the CPU 702 maybe a system-on-chip (SoC) with a multi-core processor architecture. Insome examples, the CPU 702 can be a specialized digital signal processor(DSP) used for image processing. The memory device 704 can includerandom access memory (RAM), read only memory (ROM), flash memory, or anyother suitable memory systems. For example, the memory device 704 mayinclude dynamic random access memory (DRAM).

The memory device 704 can include random access memory (RAM), read onlymemory (ROM), flash memory, or any other suitable memory systems. Forexample, the memory device 704 may include dynamic random access memory(DRAM).

The computing device 700 may also include a digital signal processingunit (DSP) 708. As shown, the CPU 702 may be coupled through the bus 706to the DSP 708. The DSP 708 may be configured to perform any number ofaudio processing operations within the computing device 700. Forexample, the DSP 708 may be configured to measure, filter or compresscontinuous real-world analog signals corresponding to audio, or thelike, to be played back to a user of the computing device 700.

The memory device 704 can include random access memory (RAM), read onlymemory (ROM), flash memory, or any other suitable memory systems. Forexample, the memory device 704 may include dynamic random access memory(DRAM). The memory device 704 may include device drivers 710 that areconfigured to execute the instructions for processing signals fromvibration sensors and microphones generating audio for playback. Thedevice drivers 710 may be software, an application program, applicationcode, or the like.

The CPU 702 may also be connected through the bus 706 to an input/output(I/O) device interface 712 configured to connect the computing device700 to one or more I/O devices 714. The I/O devices 714 may include, forexample, a keyboard and a pointing device, wherein the pointing devicemay include a touchpad or a touchscreen, among others. The I/O devices714 may be built-in components of the computing device 700, or may bedevices that are externally connected to the computing device 700. Insome examples, the memory 704 may be communicatively coupled to I/Odevices 714 through direct memory access (DMA).

The CPU 702 may also be linked through the bus 706 to a displayinterface 716 configured to connect the computing device 700 to adisplay device 718. The display device 718 may include a display screenthat is a built-in component of the computing device 700. The displaydevice 718 may also include a computer monitor, television, orprojector, among others, that is internal to or externally connected tothe computing device 700.

The computing device 700 also includes a storage device 720. The storagedevice 720 is a physical memory such as a hard drive, an optical drive,a thumbdrive, an array of drives, a solid-state drive, or anycombinations thereof. The storage device 720 may also include remotestorage drives.

The computing device 700 may also include a network interface controller(NIC) 722. The NIC 722 may be configured to connect the computing device700 through the bus 706 to a network 724. The network 724 may be a widearea network (WAN), local area network (LAN), or the Internet, amongothers. In some examples, the device may communicate with other devicesthrough a wireless technology. For example, the device may communicatewith other devices via a wireless local area network connection. In someexamples, the device may connect and communicate with other devices viaBluetooth® or similar technology.

The computing device 700 further includes a set of safety glasses 726with vibration sensor 728. For example, the safety glasses 726 may haveextended panels to prevent objects from going behind the glasses. Insome examples, the safety glasses 726 may be made of a hi-transparencypolycarbonate hardened for high impact. In some examples, the vibrationsensor 728 may be a piezoelectric sensor or an accelerometer, amongother sensors that can detect vibrations. In some examples, thevibration sensor 728 may be located on the nose pad of the safetyglasses 726. The vibration sensor 728 can capture vibrations at a nosethat are associated with speech. For example, the vibrations at the nosemay be caused by a user speaking.

The computing device 700 further includes a hearing protectors 730. Thehearing protectors 730 may include a microphone 732 and a speaker 734.For example, the microphone 732 may be used to capture ambient soundfrom the environment of the hearing protectors 730. The speaker 734 maybe used to play back audio. In some examples, the audio may includeambient sounds with reduced volume, notifications, speech from otherdevices, or TTS responses, among other types of audio.

The computing device 700 further includes an integrated hearingprotector and communicator (IHPC) 731. For example, the IHPC 731 can beused to protecting hearing and enable communication in environments withloud noise. For example, an environment with loud noise may includenoise with an amplitude of 85 dBA or more. The IHPC 731 can include avibration recorder 736, a speech processor 738, a speech transmitter740, a receiver 742, an event detector 744, a loudness equalizer 746, asound logger 748, and an audio mixer 750. In some examples, each of thecomponents 736-750 of the IHPC 731 may be a microcontroller, embeddedprocessor, or software module. The vibration recorder 736 can capturespeech from a user. The speech processor 738 can process the speechbased on an active voice call. For example, the speech processor 738 canprocess the captured speech with an automatic speech recognition (ASR)voice transformation in response to not detecting an active voice call.In some examples, the speech processor 738 can process the capturedspeech with a human-to-human voice transformation and an ASRtransformation in response to detecting an active voice call. Forexample, a destination of the active voice call may be an external IHPCor other device that can play back audio for human consumption. Thespeech transmitter 740 can transmit the captured speech to one or moreother devices. In some examples, the speech transmitter 740 can transmitthe captured speech to one or more other devices based on a detecteddestination of the speech. For example, the speech transmitter 740 cansend the speech to one or more devices based on a detected keywordassociated with the one or more devices. The receiver 742 can receiveprocessed speech feedback received from other devices. The eventdetector 744 can detect an audio event in the ambient sound and log theaudio event. For example, the audio event may correspond to alarms,sudden loud noises or glass breaking. In some examples, the eventdetector 744 can generate a notification based on the detected audioevent. The loudness equalizer 746 can reduce a volume of an ambientsound. For example, the loudness equalizer 746 can equalize the loudnessof the captured ambient sounds based on a target loudness.

The sound logger 748 can log ambient sounds. For example, the loggedambient sounds can be used to can be used to provide means to measurenoise exposure of users. The audio mixer 750 can combine the processedambient sounds, processed speech feedback received from other devices,and the notifications. For example, the combined processed ambientsounds, processed speech feedback received from other devices, andnotifications can then be played back on a device, such as the speakers734 of the hearing protectors 730.

The computing device 700 can further be communicatively coupled to oneor more external devices 752. In some examples, the computing device 700can send and receive processed audio to the external device 752. Forexample, the external devices 752 may be integrated hearing protectionand communication devices.

The block diagram of FIG. 7 is not intended to indicate that thecomputing device 700 is to include all of the components shown in FIG.7. Rather, the computing device 700 can include fewer or additionalcomponents not illustrated in FIG. 7, such as additional buffers,additional processors, and the like. The computing device 700 mayinclude any number of additional components not shown in FIG. 7,depending on the details of the specific implementation. Furthermore,any of the functionalities of the vibration recorder 736, the speechprocessor 738, the speech transmitter 740, the receiver 742, the eventdetector 744, the loudness equalizer 746, the sound logger 748, and thesound mixer 750, may be partially, or entirely, implemented in hardwareand/or in the processor 702. For example, the functionality may beimplemented with an application specific integrated circuit, in logicimplemented in the processor 702, or in any other device. In addition,any of the functionalities of the CPU 702 may be partially, or entirely,implemented in hardware and/or in a processor. For example, thefunctionality of the IHPC 731 may be implemented with an applicationspecific integrated circuit, in logic implemented in a processor, inlogic implemented in a specialized audio processing unit such as the DSP708, or in any other device.

FIG. 8 is a block diagram showing computer readable media 800 that storecode for providing communication and ear protection in noisyenvironments. The computer readable media 800 may be accessed by aprocessor 802 over a computer bus 804. Furthermore, the computerreadable medium 800 may include code configured to direct the processor802 to perform the methods described herein. In some embodiments, thecomputer readable media 800 may be non-transitory computer readablemedia. In some examples, the computer readable media 800 may be storagemedia.

The various software components discussed herein may be stored on one ormore computer readable media 800, as indicated in FIG. 8. For example, avibration recorder module 806 may be configured to capture speech from avibration sensor of a first device. For example, the first device may bean integrated hearing protection and communication device. A speechprocessor module 808 may be configured to process the captured speechbased on an active voice call. In some examples, the speech processormodule 808 may be configured to process the captured speech with anautomatic speech recognition (ASR) voice transformation and ahuman-to-human voice transformation in response to detecting an activevoice call. In some examples, the speech processor module 808 may beconfigured to process the captured speech with an automatic speechrecognition (ASR) voice transformation in response to not detecting anactive voice call. In some examples, the speech processor module 808 maybe configured to perform a keyword or ASR recognition on the processedspeech and sending the recognition results to a natural languageprocessing (NLP) system. A speech transmitter module 810 may beconfigured to transmit the captured speech to one or more other devices.For example, the speech transmitter module 810 may be configured to sendthe captured speech to a device based on a detected keyword. A receivermodule 812 may be configured to receive ambient sounds from a microphoneof a first device. In some examples, the receiver module 812 may beconfigured to receive a text-to-speech (TTS) response from an NLPsystem. An event detector module 814 may be configured to detect audioevents in the captured audio sounds and log the events in an event log.In some examples, the event detector module 814 may be configured togenerate notification sounds based on the detected audio events. Forexample, the notification sounds may be hazard notifications, amongother types of notifications. A loudness equalizer module 816 may beconfigured to process the ambient sounds to generate ambient sounds withlower volume. For example, the loudness equalizer module 816 may beconfigured to equalize the loudness of the captured ambient sounds basedon a target loudness. For example, the target loudness may be presentand user adjustable. A sound logger module 818 may be configured tocharacterize and log the ambient sounds. For example, the sound loggermodule 818 may include a neural network trained to characterize theambient sounds. An audio mixer module 820 may be configured to combineprocessed ambient sounds, processed speech feedback received from otherdevices, and the notifications, for playback. The combined processedambient sounds, processed speech feedback received from other devices,and the notifications can then be played back on speakers of anintegrated hearing protection and communication device.

The block diagram of FIG. 8 is not intended to indicate that thecomputer readable media 800 is to include all of the components shown inFIG. 8. Further, the computer readable media 800 may include any numberof additional components not shown in FIG. 8, depending on the detailsof the specific implementation.

EXAMPLES

Example 1 is an apparatus for hearing protection and communication. Theapparatus includes safety glasses including a vibration sensor tocapture speech from a user. The apparatus includes hearing protectorscommunicatively coupled to the safety glasses and one or more otherdevices. The hearing protectors are to reduce a volume of an ambientsound and play back captured speech from the one or more other devices.The apparatus includes a plurality of wireless communication elements tocommunicatively couple the safety glasses, the hearing protectors, and asecond apparatus for hearing protection and communication.

Example 2 includes the apparatus of example 1, including or excludingoptional features. In this example, the wireless communication elementsinclude short-range devices.

Example 3 includes the apparatus of any one of examples 1 to 2,including or excluding optional features. In this example, vibrationsensor includes a piezoelectric sensor or an accelerometer.

Example 4 includes the apparatus of any one of examples 1 to 3,including or excluding optional features. In this example, the vibrationsensor is integrated into a nose pad of the safety glasses.

Example 5 includes the apparatus of any one of examples 1 to 4,including or excluding optional features. In this example, the hearingprotectors include a microphone to capture the ambient sound.

Example 6 includes the apparatus of any one of examples 1 to 5,including or excluding optional features. In this example, the hearingprotectors include a speaker to playback audio including the ambientsound with reduced volume and the captured speech.

Example 7 includes the apparatus of any one of examples 1 to 6,including or excluding optional features. In this example, the hearingprotectors are to further generate a notification based on the ambientsound.

Example 8 includes the apparatus of any one of examples 1 to 7,including or excluding optional features. In this example, the hearingprotectors are to detect an audio event in the ambient sound and log theaudio event.

Example 9 includes the apparatus of any one of examples 1 to 8,including or excluding optional features. In this example, the hearingprotectors include a voice transformer to process the speech based on adetected active voice call.

Example 10 includes the apparatus of any one of examples 1 to 9,including or excluding optional features. In this example, the hearingprotectors are to detect a target device to send the captured speechbased a detected destination of the speech, the target device includingthe second apparatus for hearing protection and communication.

Example 11 is a method for hearing protection and communication. Themethod includes capturing speech from a vibration sensor and ambientsounds from a microphone of a first device. The method also includesprocessing, via the processor, the captured speech based on an activevoice call. The method further includes processing the ambient sounds togenerate ambient sounds with lower volume and notifications based ondetected sound events. The method also further includes playing back theprocessed ambient sounds, processed speech feedback received from otherdevices, and the notifications.

Example 12 includes the method of example 11, including or excludingoptional features. In this example, the method includes transmitting theprocessed speech to the other devices and receiving the processed speechfeedback from one or more of the other devices.

Example 13 includes the method of any one of examples 11 to 12,including or excluding optional features. In this example, processingthe captured speech includes processing the captured speech with anautomatic speech recognition (ASR) voice transformation and ahuman-to-human voice transformation in response to detecting the activevoice call.

Example 14 includes the method of any one of examples 11 to 13,including or excluding optional features. In this example, processingthe captured speech includes processing the captured speech with anautomatic speech recognition (ASR) voice transformation in response tonot detecting the active voice call.

Example 15 includes the method of any one of examples 11 to 14,including or excluding optional features. In this example, the methodincludes performing a keyword or ASR recognition on the processed speechand sending the recognition results to a natural language processing(NLP) system.

Example 16 includes the method of any one of examples 11 to 15,including or excluding optional features. In this example, the methodincludes receiving a text-to-speech (TTS) response from the NLP systemand playing back the TTS response.

Example 17 includes the method of any one of examples 11 to 16,including or excluding optional features. In this example, the methodincludes sending the captured speech to a device based on a detectedkeyword.

Example 18 includes the method of any one of examples 11 to 17,including or excluding optional features. In this example, the methodincludes detecting audio events in the captured audio sounds and loggingthe events in an event log and generating notification sounds based onthe detected audio events.

Example 19 includes the method of any one of examples 11 to 18,including or excluding optional features. In this example, the methodincludes equalizing the loudness of the captured ambient sounds based ona target loudness.

Example 20 includes the method of any one of examples 11 to 19,including or excluding optional features. In this example, the methodincludes characterizing and logging the ambient sounds.

Example 21 is at least one computer readable medium for hearingprotection and communication having instructions stored therein thatdirect the processor to capture speech from a vibration sensor andambient sounds from a microphone of a first device. Thecomputer-readable medium also includes instructions that direct theprocessor to process the captured speech based on an active voice call.The computer-readable medium further includes instructions that directthe processor to transmit the captured speech to one or more otherdevices. The computer-readable medium also further includes instructionsthat direct the processor to play back the processed ambient sounds,processed speech feedback received from other devices, and the soundevents.

Example 22 includes the computer-readable medium of example 21,including or excluding optional features. In this example, thecomputer-readable medium includes instructions to process the ambientsounds to generate ambient sounds with lower volume and sound events.

Example 23 includes the computer-readable medium of any one of examples21 to 22, including or excluding optional features. In this example, thecomputer-readable medium includes instructions to process the capturedspeech with an automatic speech recognition (ASR) voice transformationand a human-to-human voice transformation in response to detecting theactive voice call.

Example 24 includes the computer-readable medium of any one of examples21 to 23, including or excluding optional features. In this example, thecomputer-readable medium includes instructions to process the capturedspeech with an automatic speech recognition (ASR) voice transformationin response to not detecting the active voice call.

Example 25 includes the computer-readable medium of any one of examples21 to 24, including or excluding optional features. In this example, thecomputer-readable medium includes instructions to perform a keyword orASR recognition on the processed speech and sending the recognitionresults to a natural language processing (NLP) system.

Example 26 includes the computer-readable medium of any one of examples21 to 25, including or excluding optional features. In this example, thecomputer-readable medium includes instructions to receive atext-to-speech (TTS) response from the NLP system and playing back theTTS response.

Example 27 includes the computer-readable medium of any one of examples21 to 26, including or excluding optional features. In this example, thecomputer-readable medium includes instructions to send the capturedspeech to a device based on a detected keyword.

Example 28 includes the computer-readable medium of any one of examples21 to 27, including or excluding optional features. In this example, thecomputer-readable medium includes instructions to detect audio events inthe captured audio sounds and log the events in an event log andgenerating notification sounds based on the detected audio events.

Example 29 includes the computer-readable medium of any one of examples21 to 28, including or excluding optional features. In this example, thecomputer-readable medium includes instructions to equalize the loudnessof the captured ambient sounds based on a target loudness.

Example 30 includes the computer-readable medium of any one of examples21 to 29, including or excluding optional features. In this example, thecomputer-readable medium includes instructions to characterize and logthe ambient sounds.

Example 31 is a system for hearing protection and communication. Thesystem includes safety glasses including a vibration sensor to capturespeech from a user. The system includes hearing protectorscommunicatively coupled to the safety glasses and one or more otherdevices. The hearing protectors are to reduce a volume of an ambientsound and play back captured speech from the one or more other devices.The system includes a plurality of wireless communication elements tocommunicatively couple the safety glasses, the hearing protectors, and asecond set of safety glasses and hearing protectors for hearingprotection and communication.

Example 32 includes the system of example 31, including or excludingoptional features. In this example, the wireless communication elementsinclude short-range devices.

Example 33 includes the system of any one of examples 31 to 32,including or excluding optional features. In this example, vibrationsensor includes a piezoelectric sensor or an accelerometer.

Example 34 includes the system of any one of examples 31 to 33,including or excluding optional features. In this example, the vibrationsensor is integrated into a nose pad of the safety glasses.

Example 35 includes the system of any one of examples 31 to 34,including or excluding optional features. In this example, the hearingprotectors include a microphone to capture the ambient sound.

Example 36 includes the system of any one of examples 31 to 35,including or excluding optional features. In this example, the hearingprotectors include a speaker to playback audio including the ambientsound with reduced volume and the captured speech.

Example 37 includes the system of any one of examples 31 to 36,including or excluding optional features. In this example, the hearingprotectors are to further generate a notification based on the ambientsound.

Example 38 includes the system of any one of examples 31 to 37,including or excluding optional features. In this example, the hearingprotectors are to detect an audio event in the ambient sound and log theaudio event.

Example 39 includes the system of any one of examples 31 to 38,including or excluding optional features. In this example, the hearingprotectors include a voice transformer to process the speech based on adetected active voice call.

Example 40 includes the system of any one of examples 31 to 39,including or excluding optional features. In this example, the hearingprotectors are to detect a target device to send the captured speechbased a detected destination of the speech, the target device includingthe second set of safety glasses and hearing protectors for hearingprotection and communication.

Example 41 is a system for hearing protection and communication. Thesystem includes means for capturing speech from a user. The system alsoincludes means for reducing a volume of an ambient sound and playingback captured speech from the one or more other devices communicativelycoupled to the means for capturing speech from a user and one or moreother devices. The system further includes means for communicativelycoupling the means for capturing speech from the user, the means forreducing the volume of the ambient sound and playing back capturedspeech, and a second set of means for capturing speech from the user andmeans for reducing the volume of the ambient sound and playing backcaptured speech.

Example 42 includes the system of example 41, including or excludingoptional features. In this example, the means for communicativelycoupling include short-range devices.

Example 43 includes the system of any one of examples 41 to 42,including or excluding optional features. In this example, means forcapturing speech includes a piezoelectric sensor or an accelerometer.

Example 44 includes the system of any one of examples 41 to 43,including or excluding optional features. In this example, the means forcapturing speech is integrated into a nose pad of the safety glasses.

Example 45 includes the system of any one of examples 41 to 44,including or excluding optional features. In this example, the means forreducing the volume of the ambient sound and playing back the capturedspeech include a microphone to capture the ambient sound.

Example 46 includes the system of any one of examples 41 to 45,including or excluding optional features. In this example, the means forreducing the volume of the ambient sound and playing back the capturedspeech include a speaker to playback audio including the ambient soundwith reduced volume and the captured speech.

Example 47 includes the system of any one of examples 41 to 46,including or excluding optional features. In this example, the means forreducing the volume of the ambient sound and playing back the capturedspeech are to further generate a notification based on the ambientsound.

Example 48 includes the system of any one of examples 41 to 47,including or excluding optional features. In this example, the means forreducing the volume of the ambient sound and playing back the capturedspeech are to detect an audio event in the ambient sound and log theaudio event.

Example 49 includes the system of any one of examples 41 to 48,including or excluding optional features. In this example, the means forreducing the volume of the ambient sound and playing back the capturedspeech include a voice transformer to process the speech based on adetected active voice call.

Example 50 includes the system of any one of examples 41 to 49,including or excluding optional features. In this example, the means forreducing the volume of the ambient sound and playing back the capturedspeech are to detect a target device to send the captured speech based adetected destination of the speech, the target device including thesecond set of means for capturing speech from the user and means forreducing the volume of the ambient sound and playing back capturedspeech.

Not all components, features, structures, characteristics, etc.described and illustrated herein need be included in a particular aspector aspects. If the specification states a component, feature, structure,or characteristic “may”, “might”, “can” or “could” be included, forexample, that particular component, feature, structure, orcharacteristic is not required to be included. If the specification orclaim refers to “a” or “an” element, that does not mean there is onlyone of the element. If the specification or claims refer to “anadditional” element, that does not preclude there being more than one ofthe additional element.

It is to be noted that, although some aspects have been described inreference to particular implementations, other implementations arepossible according to some aspects. Additionally, the arrangement and/ororder of circuit elements or other features illustrated in the drawingsand/or described herein need not be arranged in the particular wayillustrated and described. Many other arrangements are possibleaccording to some aspects.

In each system shown in a figure, the elements in some cases may eachhave a same reference number or a different reference number to suggestthat the elements represented could be different and/or similar.However, an element may be flexible enough to have differentimplementations and work with some or all of the systems shown ordescribed herein. The various elements shown in the figures may be thesame or different. Which one is referred to as a first element and whichis called a second element is arbitrary.

It is to be understood that specifics in the aforementioned examples maybe used anywhere in one or more aspects. For instance, all optionalfeatures of the computing device described above may also be implementedwith respect to either of the methods or the computer-readable mediumdescribed herein. Furthermore, although flow diagrams and/or statediagrams may have been used herein to describe aspects, the techniquesare not limited to those diagrams or to corresponding descriptionsherein. For example, flow need not move through each illustrated box orstate or in exactly the same order as illustrated and described herein.

The present techniques are not restricted to the particular detailslisted herein. Indeed, those skilled in the art having the benefit ofthis disclosure will appreciate that many other variations from theforegoing description and drawings may be made within the scope of thepresent techniques. Accordingly, it is the following claims includingany amendments thereto that define the scope of the present techniques.

What is claimed is:
 1. A wearable electronic device comprising: meansfor transducing vibrations associated with speech into a first signal;means for transducing sound associated with ambient noise into a secondsignal; and means for processing to: cause a speaker to output a signalto reduce the ambient noise; detect an identifier in the speech; andcause audio data representative of the speech to be transmitted to asecond device associated with the identifier.
 2. The wearable electronicdevice of claim 1, further including means for filtering the firstsignal.
 3. The wearable electronic device of claim 1, further includingglasses, the vibrating transducing means carried by the glasses.
 4. Thewearable electronic device of claim 1, further including headphones, thesound transducing means carried by the headphones.
 5. The wearableelectronic device of claim 1, further including means for reducing avolume of the ambient noise in response to instructions from theprocessing means.
 6. The wearable electronic device of claim 1, whereinthe processing means is to cause establishment of a communication linkwith the second device based on the identifier.
 7. The wearableelectronic device of claim 6, the processing means is to cause thespeaker to output speech received from the second device.
 8. Thewearable electronic device of claim 1, wherein the identifier isassociated with a user of the second device.
 9. The wearable electronicdevice of claim 1, wherein the processing means is to: identify a soundevent in the ambient noise; and cause the speaker to output one of thesound event or an alert indicative of the sound event in response to theidentification of the sound event.
 10. A first device comprising: atleast one memory; instructions; and at least one processor to executethe instructions to: identify a keyword in speech, the speechrepresented by signals output by a vibration sensor; cause first audiodata corresponding to the speech to be transmitted to a second devicebased on the keyword; and cause a speaker to output second audio data,the second audio data including one or more of (a) sound to cancelambient noise, (b) speech from the second device, (c) speech from athird device, or (d) a sound event different from the ambient noise andthe speech.
 11. The first device of claim 10, wherein the at least oneprocessor is to execute a neural network model to identify the soundevent in the ambient noise.
 12. The first device of claim 10, whereinthe at least one processor is to: detect the sound event in the ambientnoise; and cause the speaker to output (a) the sound to cancel ambientnoise, and (d) the sound event.
 13. The first device of claim 10,wherein the keyword includes an identifier associated with a user of thesecond device.
 14. At least one non-transitory computer readable storagemedium comprising instructions that, when executed, cause at least oneprocessor of a first device to at least: identify a sound event inambient noise, the ambient noise represented by signals output by amicrophone; cause a speaker to output an alert in response to theidentification of the sound event; identify a keyword in speech, thespeech represented by signals output by a vibration sensor; and causefirst audio data corresponding to the speech to be transmitted to asecond device or a third device based on the keyword.
 15. Thenon-transitory computer readable storage medium of claim 14, wherein theat least one processor is to cause a speaker to output a signal toreduce the ambient noise.
 16. The non-transitory computer readablestorage medium of claim 14, wherein the at least one processor is tocause the speaker to output sound to cancel the ambient noise.
 17. Thenon-transitory computer readable storage medium of claim 14, wherein theat least one processor is to cause establishment of a communication linkwith the second device or the third device based on the keyword.
 18. Thenon-transitory computer readable storage medium of claim 14, wherein thesound event is a first sound event, the ambient noise is first ambientnoise, and the at least one processor is to detect a second sound eventin second ambient noise based on the first sound event, the secondambient noise represented by signals output by a microphone after thesignals associated with the first ambient noise.
 19. The non-transitorycomputer readable storage medium of claim 14, wherein the at least oneprocessor is to cause the speaker to output speech from one of thesecond device or the third device.
 20. The non-transitory computerreadable storage medium of claim 14, wherein the keyword includes anidentifier associated with a user of the second device or the thirddevice.